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Agenda - Day 1 , ^f^SS^ 
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0815 


Check-in 


0830 


Welcome & Introduction 


0900 


NVT Program Goals and Objectives 


0930 


Roundtable 1 - Do the goals of NVT map to the community 
needs and requirements? 


1015 


Break 


1030 


Information Sharinq - RAM and DPL-f 


1100 


NVT Design - Overall Architecture 


1200 


Workinq Lunch 


1300 


NVT Demonstration - (1) Automatic Discovery 
(1) Manual Network Diagram 


1400 


NVT Design - Object Classes 


1445 


Break 


1500 


COTS Integration Lessons Learned 


1530 


Plans for Next Quarter 


1600 


Roundtable 2 - Feedback on Proof-of-Concept Prototype 


1630 


Wrap Up 
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0830 


Overview 


0845 


Protocol Mining / LANmine 


1015 


Break 


1030 


STAKEOUT 


1200 


Working Lunch 


1230 


STAT 


1430 


Briefing Center Tour 
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Program Overview 
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• Customer: AFRL/IFGB 

- LPM: DwayneAHain 

- Deputy: Peter Radesi 

• Value: $573K 

• Schedule: 24 months, 1 April 1997 start 

• Objectives 

- Investigate current risk assessment and vulnerability 
detection products to determine if they can be incorporated 
into a common framework 

- Investigate technologies for the enhancement of automated 
risk assessment technology in the areas of usability, 
productivity and capability 
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Program Overview 
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Electronic Systems 



• Objectives (continued) 

- In particular, investigate enhancement through 

• Methods to perform knowledge solicitation 

• Normalized system representation satisfying the needs of 
several existing risl< assessment tools 

• Fusion of various tool outputs into a single report 

• Graphical display of the resulting vulnerability data 

- Develop an initial Proof-of-Concept prototype 
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NVT Program Team 
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Project Manager 
Ronda Henning 



Program Administrator 
Linda Piian 



Contracts 
Eva Harris 



Principal Investigator 
Eric IMeijer 



Software Engineer 
John Farreil 



Software Engineer 
Kevin Fox 



Software Engineer 
Cliff Miller 
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Program Schedule 
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• Contract began 1 April 1997 

• 24 month schedule, divided into 7 tasks 

- Task #1 ' Knowledge Solicitation 

- Task #2 - System Visualization and Validation 

- Task #3 - Selection and Application of Automated 
Reasoning Technologies (Risk Assessment Tools) 

- Task M - Vulnerability Quantification 

- Task #5 - Scaling of Identified Vulnerabilities 

- Task #6 - Proof-of-Concept Prototype 

- Task #7 - Final Report 
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Current Status 
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• Task #1 - Knowledge Solicitation - Completed 

- Resulted in selection of OpenView as discovery technology 

- Also have NET VIZ in NVT Lab 

• Task #2 - System Visualization & Validation - On-going 

• Task #3 - Selection & Application of Automated Reasoning 
Technologies - Completed 

- Resulted in selection ofANSSR, ISS, and RAM 

• Task #4 - Vulnerability Quantification 

• Task #5 - Scaling of Identified Vulnerabilities - Underway 

- Resulted in selection of Fuzzy Expert System technology to 
integrate results from risk assessment tools 

• Task #6 - Proof-of-Concept Prototype - Underway 

- Demonstration of initial Proof-of-Concept Prototype today 

• Task #7 - Final Report 
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Milestone Schedule 
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Program A\ward 



Task 1 - Knowledge Solicitation 



Task 2 - System Visualization 



Task 3 - Automated Reasoning 



Task 4 - Vulnerability Quantification 



Task 5 - Scaling of Vulnerabilities 



Task 6 - Proof-of-Concept Prototype 



Acquire FuzzvCLIPS & Develop Ex 



AcQuire/Studv Vulnerability Tools 



Design Initial Prototype 



Decide Test Scenario/System 



Acquire Basic System Info fPemo) 



Complete Graphical User Interface 



Develop Fuzzy Expert for 3 Tools 



Integration & Test {Melbourne) 



Ship Equipment to AFRL/RRS 



Deliver. Demonstrate & Test (Rome) 



Task 7 - Final Report Preparation 



Technical Interchange Meetings 



CDRLS 



R&D Status Reports (A001R) 



CFSR fA002R) 



Presentation Materials (A003R) 



Demonstration Plan (A004R) 



Commented Source Code fAOOSR) 



Software User's Manual (A006R) 



COTS Manuals fA007R) 



Final S&T Report fAOOSR) 
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Results to date 



• Feasible to use multiple tools to fill in missing data. 

• Tools with different modes of operation can be 
combined to provide a more complete picture. 

• it is possible to combine the results of multiple tools 
into one coherent picture. 

• Fusion techniques are viable for use in report 
integration. 
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Phases of NVT 
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At our last meeting,.,. , ^fj^SHSIS 

What is Risk? 

• Risk includes: assessment, characterization, 
communication, management, and policy 
relating to risk 

• How do we quantify risk? 

• How do organizations respond to vague risks? 

- Who specifies the situational factors? 

• At what point does an organization respond to a 
risk? 

- l-low are different strategies identified and then 
deployed? 

• How is a risk response(response strategy) 
defined? 
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NVT Architecture Coals ^^^^^^ 

• Establish a framework that allows for the use of current, 
and future, vulnerability and risk assessment Plugins 
(the collective set of third party applications a user 
wishes to integrate into the NVT system) 

• The Framework shall establish the foundation for a 
system that can resolve Knowledge & Language issues 

• Provide the user with a clear understanding of their 
present risk based on the most effective use of the 
current Plug in set 

• Provide the user with the capability to determine the 
most effective means to mitigate their risk 

NVT must provide rational probable solutions 
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Today 



"Grounded" in feasible, incremental improvement 
Near term results that will have a positive impact 
Useful demonstration system 
Tangible end products 
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Your Chance to , . . 



£Sectnonie Systems 



impact the rest of our work 

Tell us what you don't like 

Tell us whether this is useful 

How we can make it more responsive 



next level solutions 



NVTTIM #6.# 



Electronic Systems 



Roundtable 1 

Do the goals of NVT map to 
the community needs and 
requirements? 
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information Sharing ~ 
OAH/r and DPL-f 

Capt. Don Buckshaw 
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NVT Design 

Architecture 
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NVT Architecture Goals 



• Establish a framework that allows for the use of 
current and future risk assessment plugins 

• Establish the foundation for a system that can 
resolve Ontological and Language issues 

• Provide the user with a clear understanding of their 
present risk based on the most effective use of the 
current plugin set 

• Provide the user with the capability to determine the 
most effective means to mitigate their risk 
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NVT Architecture Concept 
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NVT Core Components 




Plugins 



Problem \ 
Domain V 
1^ Specific ^ 

■ Applications^ 
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Technology Assessment 



y/^ Etectronic Systems 



• Graphical User Interface (GUI) 

- Determine the GUI, do not invent new visualization 
techniques, but focus on applying work already done in 
other related areas, such as data fusion, message 
understanding, virtual reality, etc. 

- Understand that this probably at least a two part GUI, one 
for input and one for output 

• Plugin Autonomous Control (fusion) 

- Focused on technologies to support automated integration 
of output from multiple risk assessment tools 

- Selected Fuzzy Expert System 

• Fuzzy expert systems use a collection of fuzzy membership 
functions and rules, instead of Boolean logic, to reason about 
data 
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COTS & GOTS Products , ^^l^fHg 

• Visualization 

- HPOpenView 

- NetViz 

• Processing 

- MS Visual C++ (Visual Studio) 

- Smalltalk (for use with ANSSR) 

• Storage 

- MS Access database 

- Oracle 

• Plugin Autonomous Control 

- FuzzyCLIPS - an extension of standard CLIPS that allows 
for the use of fuzzy facts and fuzzy rules which contain both 
membership functions and certainty factors 



next level solutions 



NVTTIM#6,# 



Risk Analysis Tools 
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• The integration of three distinct risk analysis/ 
vulnerability analysis reasoning engines Into a proof- 
of-concept prototype 

• One tool was chosen to represent each of the 
different categories of vulnerability tools 

- ANSSR was selected as a prime example of a legacy 
reasoning engine 

- ISS Intemet Scanner was selected as an example of a 
"live" vulnerability tool 

- RAM was selected because of our experience using it for 
large scale, highly complex problems such as the power 
distribution system and because it was selected for the 
Secret and Below Initiative (SABI) Risk Analysis 
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HP OpenView 



&ectronic Systems 



• Automatic Discovery 

- 6/Ven \P address of the default router, it searcfies for 
computers & other devices attached to the networl< 

- Performs an active search, pinging possible IP addresses 
on the network 

• Adds \whatever response info it receives to its networl< map 

• Manual Network Diagram 

- Provides a method to draw a proposed network 

- Properties of each network node can be edited 

• Add details to provide complete logical network planning 

- Can represent an entire network on a map by using a 
subnetwork icon 

• Detailed map of the subnetwork can be linked to this icon and 
be displayed by double-clicking the subnetwork icon 
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jNetyprKVi$w; lQ.o.0,0 

Discovered Noder 





10. e. 0.1 


Generic IP Deuice 




10. e. 0.18 


Generic IP Deuice 




10. 0.0.100 


Generic IP Device 




10.0.0.15 


Generic IP Device 


i 


10.0.0.2 


Generic IP Device 




10.0.0.3 


Generic IP Device 




10.0.0.4 


Generic IP Device 






ANSSR HAJUUS 

• Data Assets 

- Primary object used in risk assessment 

- Vulnerability of assets across network and through the user 
communities 

- Represents protection mechanisms 

• Use 

- Requires Smalltalk 

- Manual Entry through numerous menus 

- Textual representation of Network 
• No visual representation 

- Final risk assessment is a text file 
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iSS internet Scanner 
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• Scans existing network for vulnerabilities 

- Good assessment of system configuration errors 

- Information for system and network configuration may be 
used to setup ANSSR models 

- Input policy and session files control the scan 

• Maintains scan results in Access database 

- Delivered with capability to export directly to Microsoft 
Access 

- Scan results easily imported to NVT through SQL 

• Has discovery capabilities 

- Appears more reliable then OpenView 
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Prototype GOTS/COTS 
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NVT Core Components 




Plugins 
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Working Lunch 
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Demonstration 

NVT 

Proof-of-Concept 
Prototype 
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Prototype Demonstration , ^^l^SfB^ 

• Automatic Discovery 

• IVianual Network Diagram 
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NVT Design 

Object Classes 
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NVT Prototype Architecture , 
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NVT Message Trace 



Electronic Systems 



ftieMVTGui: 
NVTDsbgs 



ftieFxizzyFusjpn : 



ClpsIitBi&ce 



18 rasseiJFactsO 

19:im() 
20:iEtCEveFacts( ) 



4:NetwoijtObpct{ ) 
6:addChiH() 



aSystBiitNode : 



NetwoAOb^ct 



listaitO 



ftifiNVTOb^ct: 



NVTOb:gct 



5:Netwo3kOb:pct() 



aLiik : 
NetwoikOb:pct 



2:stait:() 
3 :gea)ata t^elwoikData) 
10:e3port^ ) 
14:ga&ie£i ata () 
> 



7:compfeteDafe 0 
ll:e3poit() 
15:galheiData() 



8:compfetBData() 

12:e)port() 
16:gaftifiiDala() 
> 



9:comp]eteData 0 

13:e3port() 
17:gaftieS)afe{) 
> 



ftieO VFiter: 



OVFiter 



theANSSRFiter: 
ANSSRFilter 



freESFiter: 



ESFJter 



ftieRAMFiter: 



RAM Filter 
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Required Classes forANSSR 
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Locaaon | | Pereon Data Assets KtehwotMntefface 



Accraditatia^ange Sef>rtceProvided KamadOata SyaemAidiitacture 



ANSSR NatvkO ((Component 



Central Seivicq 



IdentifiedElement 



1 



IdentifiedPhyacalBement 



SydemNode 


x> 


Node 









InputChannel 












OutpulChannel 









•j^ lOChannri 



Comp09 ti onat Descri pti on 


AppI icationSystem Attri bu te« 







ri7 

_J Clearai 



Communi call on System Attri bute» 



EncryptionChafaclerisllcOfSeed 







1 








UsefCommunity 










UaintainerCommunity 






OperatorCommuni ty 















Seed 




1 












EncryptionCharacteristies 
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Class Interaction forANSSR 
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NVTOb^Ct 



^IheO VF SiBT :0 VF iter 
[^tie B SF iter : E SF iter 
l^ftieAN S S R F iter :AN S SR Fitel 
[^^tieNetworic iNetworicOb^ct 
t^^ftieR AM F iter :R AM F iter 



:vDia 



-ftieESFiter — 
— aieANSSRFitert 



ftieR AMFiter 



ANSSRFiter 







NetwodcOb^ct 









RAMFiter 



NetooAData 



CunentV cooiEsponds 
wiiANSSRNetwoA 
C om ponent 



U sed (hmughoutN VT fcr 
Data RepiEsentatbn Abng 
w^fiteis,ftus albws a 
siigfe arshiBCtuiE to talt to 
m an/toofe 



(l{>ftieR esute Q :N etvo^^ ata 



^^jgetData 0 
rj^puOaa 0 
l^e3T)OJi:0 
l;5^aftiejData 0 



NVTDibgs 












Used ftUDughoutNVT 
£>rany U ser input 
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Node Representation 



EBectronie Systems 



IdenttfledPtiysicaiElement 



EnciyptionChafacteristics 



theE ncrypti onCharacterist) cs 



Node 



theBackupTo : Node 
theAtlowsQalln : bool 
theNetworldnterfaces : Netwoiklntefface 
theAccreditallonRange : AcoBditation Range 
theSofvicesProvidedToThisNbde : ServiceProvided 



O-theNetworMnterfaces— ^ theEi 



NetworHnteiface 



the In putAccre dilation Range ; AccreditationRange 
theOutputAccredltati on Range : AccreditationRange 

Incryption Characteristics : EncryptionCharacteristics 
theChannelsIn : lOChannel 
theCtiannelsOut : lOChannel 



UheAccieditationRange 
th eSe tvi cesProvi ded ToTti i sNode 



ServiceProvided 



theSen^iceProviders: Node 
IheAltemative Action : enum 



the In put Accred i tationRar^ge 

-theOutputAccteditati on Range 



theChannelsOut 



theChannelstn 



AccreditationRange 



theAccreditationRange : SensitivityLevel 



tOChannel 



theAccreditationRange : AccreditationRange 



-theAccreditationRange- 



CentialSer/ice 



theAccreditationRange 



SensitivityLevel 



theHLevel : ^smbol 
theCatagories : symbol 
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System Node 



SystemAfchitBctire 



-th« System Archttecture- 



SytbrnNodB 



li«Mode010p6ration : enum 

t>»CommuficationA»ributes: CoftimunicatiaiSydomAttributo* 
raSydernArttiitectire : SyetemArchitftcbf « 



O-thftCommunicati onAttributes-^ 



CommunicatiooS>a<niAltiliJtet 



ttwApplicationAttribataa 



Appli cat! on Syria mAttribotea 



IheSecifityfVoila : enum 
the (JssrCbn¥nuni ties : U»rCommunity 
theCustaneCommtriiies : Community 
iheDataAssets: DataAssets 



(he UserCbmmuniti ee— 



UsB (Community 





Community 


, > 

-ttieCustomerCommuni tie i 





the Data Assets— 



heOsdoajreValue :tlost 
helnteg-ityVaiue : float 
(heValuelnherent : boolean 
lieReplenishmentAndCorrftctionRate : float 
tieSpedalFYotection : enum 
tteDuplicates : DataAssets 
theVolume : int 

lieMaximumAcoeptiibieOutaseDurxtion: float 
Ih etmpatanca OC one ctness: enum 
helmpcrtanceOlntemalConsistency ; enum 
tielmpatanceOCompleteness : enun 
theSenstivi^ : SenstivityLevel 



O — theSensitivity — > 



ttieHLevel : sysmbd 
UieCatagories : symbol 
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identified Pitysicai Eiements 
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BentiledElement 



thetOCDate : NamedDate 
theEndDate : NamedDate 
theMissions : Mission 



-theMissions- 



Mission 



theSuperMissions : Mission 
theSubMissbns : Missbn 





Location 




theLjocation : int 


^ 





IdentifiedPhysicalElement 



thePhysicaProtection : enum 
IheMalntenanceEnMronment : enum 
IheDevelopmentEn^nonment : enum 
theDaa : CString 
theLocatton : Location 

theOperatorCommunity : OperatoiCommunity 
theMaintainetCommunity : MaintainerComm unity 
ttieDeveloperComm unity : Community 



-ttwLocation-' 



-theOperatoiComm unity- 



-ttieDevelopeiComm unity - 



ttieMai ntatnerCommunity 



OperatOfComm unity 



Community 



ttieNumbeiOfTotal ; int 
theSubcommunities : Community 
theleastCleatBnce : Clearance 
theMembeshp : Person 
theMembersliip : Person 



1 5, 


MaintaineiComm unity 


ttieTwoPersonRule : bool 
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Community Representation 
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Community 



theNumberOffotat : int 
theSubcommunities : Community 
theLeastaearaixie : Qearance 
theMemiaeship : Person 
theMembership : Person 



SensltlNityLevel 



theHLevel : sysmbol 
theCatagories : symbol 



-theLeastClearanc e- 



I 



Clearance 



-theMembeiship- 



Person 



UserCommunity 



theNumberOfOnline : int 
theLocalProcessingCapabiliy : enum 
theCommPath : enum 
theUserCapability : erum 



MaintainerCommunity 



theTwoPersonRule : bool 



OperatorCommunity 
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COTS Integration 
Lessons Learned 

1 
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HP OpenView 



etectronic Systems 



• Supplies extensive API, not always a good thing 

- C++ API several versions behind current MS compiler 

• Limited 32-bit support 

- OpenView designed for embedding and executing 
compliant applications from within OpenView itself 

- Communicating with OpenView from an external 
application not well supported 

• Final communications required files 

- Open View API not thread safe. 

- Full duplex communications through available IPC not 
possible with the OpenView API 

- Network received from OpenView through a file 

• Auto discovery quirks in the NVT Lab environment 
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AUSSR 




• Designed to run standalone, therefore no supplied 
API 

• Required version of Smalltalk no longer available 

- Encountered challenges in integrating this tool under 
Visual Smalltalk due to Smalltalk compatibility issues 

- ANSSR 2.2 written in ObjectWorks Smalltalk version 4.1 

- Acquired alternate Smalltalk (Object Share VisualWorks 
3.0, a readily available successor to ObjectWorks) 

• Solved most compatibility issues 

• ANSSR has now been successfully built under VisualWorks 
3.0 

• ANSSR delivered with code 

- ANSSR has now been further modified under VisualWorks 
to integrate with NVT 
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ANSSR Integration Issues 
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ObjectWorks 
Smalltalk v4.1 



Evaluation Copy 



VisualWorks 
Smalltalk v3.0 



NVTLab 



Visual 
Smalltalk v3.11 




Image 



Classes, 
' Methods, Code, ] 
^ Minor Portingy 
Issues 



Code, etc., 
fMore Significanty 
JPorting Issues/ 



NVT Lab 



ANSSR 

v2-2 



/1^ew\ 
Xjmap/ 
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iSS internet Scanner . 'fl^SS^ 



• Purchased, installed in NVT Lab, and tested on the 
test networli 

• No major problems associated with integration and 
use in NVT are anticipated at this time 



next level solutions 



NfVrTIM#6,# 



Oracle originally selected 

- May need to eventually use for two reasons 

• Scaling to larger networks (e.g. bases, multiple bases fonning 
a command) 

• Object representation (Oracles Object-Relational extension) 
Switched to MS Access 

- Ease of Use 

- Conr)patibility with ISS Internet Scanner 

- Excellent DAO support with Visual C++ 
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RAM 
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• Harris/NSA working CRADA for use of RAM in NVT 

- Pending NBA legal review 

• Current plan is to use the Excel spreadsheet version 

- Stable 

- Good experience base at NSA (R52/P5) 

- Eliminates procurement lead time 

• Applied Decision Analysis (ADA) building it into DPL-f 

• Nobody at ADA has a price for the product 

- RAM spreadsheet is recognizable 

- Used in SABI Risk Analysis Assessment 

• However, may need to consider DPL-f 
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FuzzyCLIPS 



• No major issues yet 

• Seems to integrate well with Visual C++ based on 
testing performed to-date 
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Plans for Next Quarter 
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Next Quarter 



hiATUZtS 

Electronic Sys^ta 



• Complete the Initial NVT Proof-of-Concept Prototype 

- Enhance the ANSSR l/F to catch more of what is there 

- Get CRADA with NSA completed to acquire RAM 

- Integrate RAM 

- Develop Fuzzy Expert to perform to fusion on outputs from 
our 3 r/s/c assessment tools 

- Complete Graphical User Interface (GUI) 
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HP OpenView 



• Network Node Evaluation 

- Show attack probabilities and vulnerabilities for any node 
on a network, even a subnetwork 

- Provide methods for the user to describe the types of 
attacks and security risks that are of concern 

- Allow user to fine-tune this information for various nodes 
on the network as well as establish a default value for the 
network 

• This fine-tuning provides a greater level of detail for 
FuzzyCLIPS to provide a more accurate summary of the risk 
assessment 
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Next Quarter 
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• Integration and Test (in IVIeibourne) 

- The final integration/demonstration testing, that determines 
removal and/or documentation of the problems 

• Ship Equipment to AFRL/RRS 

- Pack it all up, and send it out 

• Deliver, Demonstrate and Test (in Rome) 

- Take it to Rome and make it happen for final sell-off 

• Complete tlie documentation 

- Demonstration Plan 

- Software User's Manual 

- Final Scientific & Technical Report 
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Future Plans 

Beyond NVT 
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Future Enhancements 



Electronic Systesns 



• Replace HP OpenView as the Visualization tool 

- Single interface for entry of data for use by multiple 
reasoning engines 

• Incorporate Static Vulnerability Database(s) 

- SEI's CERT and Harris STAT 

- More comprehensive vulnerability analysis of a system witti 
respect to known vulnerabilities 
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Future Enhancements 
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• Vulnerability Thresholding 

- Minimizes continued computation when an aggregate 
vulnerabiiity level exceeds a user defined limit 

- Allows user to define own vulnerability tolerance level, 
which supports tailorable definitions of acceptable levels of 
vulnerability. 

• Knowledge Translation Ontology 

- Provides a common frame of reference for all tools & DBs 

- Facilitates deriving knowledge collected from other 
applications and existing tools, addressing problem of 
incomplete data for a given vulnerability assessment 
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Future Enhancements 
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• Temporal-based Reasoning and Vulnerability 
Modeling 

- Accounts for time required to exploit a l<nown vulnerability 
as part of the system assessment process 

- Enable user to perform a vulnerability assessment that 
takes into account the time required to exercise a given 
vulnerability 

• Vulnerability Trade-off Visualization 

- Use n-dimensional visualization technology 

- Allow user to perform what-if optimizations among 
performance, functionality, and countermeasures 
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3D Visualization 



• VisualEyes 

- SGI Platform 

- View 3-D and n-D data sets 

- Information Retrieval application 

- Open platform 

• RiskA/ulnerability Trade-off Analysis 

- A system architecture is assigned values for security, 
functionality, performance, availability and survivability 

- Display similar to text retrieval 

• Cube represents a particular architecture design 

• Two 3D views displayed simultaneously 

- Security, functionality and performance 

- Security, availability and survivability 
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Roundtable 2 

Feedback on Initial 
NVT Demo 
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Wrap Up 

1 

Click to add sub-title 
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Action Items 



• Open Action Items 

- HAI #1 1: Provide AFRL with the cost breakout per tool 
option for NVT and the NVT core elements 



HARDWARE COST 

NT Workstation $5130 
SOFTWARE 

FuzzyCLIPS Freeware 

HP OpenView $950 

MS Visual C++ $495 

IVIS Access DB $299 

Visual Works 3.0 (SmallTalk) $2375 

ANSSR GFI 

ISS Internet Scanner $2975 
(30 user minimum license) 

RAM GFI 

TOTAL $12,224 
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Action Items 
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• Open Action Items 

- HAl #9: Provide AFRL with a Harris IP address to facilitate 
transfer of AFIWC Vulnerability/Threat data 

_ RAI #4: AFIWC to work through Dwayne Main to provide 
access to their vulnerability/risk assessment tools 

- RAI #5; Dwayne Allain to investigate providing the CTAPS 
Air Tasking Mss/on Planning video as illustration of the 
battle/attack planning process 
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Action Items , ^^liSSL 



• New Action Items ? 
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